Neural information retrieval (IR) systems have progressed rapidly in recent years, in large part due to the release of publicly available benchmarking tasks. Unfortunately, some dimensions of this progress are illusory: the majority of the popular IR benchmarks today focus exclusively on downstream task accuracy and thus conceal the costs incurred by systems that trade away efficiency for quality. Latency, hardware cost, and other efficiency considerations are paramount to the deployment of IR systems in user-facing settings. We propose that IR benchmarks structure their evaluation methodology to include not only metrics of accuracy, but also efficiency considerations such as a query latency and the corresponding cost budget for a reproducible hardware setting. For the popular IR benchmarks MS MARCO and XOR-TyDi, we show how the best choice of IR system varies according to how these efficiency considerations are chosen and weighed. We hope that future benchmarks will adopt these guidelines toward more holistic IR evaluation.
translated by 谷歌翻译
尽管对抽象中的英语句子进行了广泛的研究,但是通过自动度量标准与金图相比,它与金图类进行了比较,但是统一图表表示的全文解析缺乏定义明确的表示和评估。利用以前的工作中的超级信托级别注释,我们介绍了一种用于导出统一图形表示的简单算法,避免了从合并不合并和缺乏连贯性信息丢失的陷阱。接下来,我们描述了对Swatch度量标准的改进,使其易于进行比较文档级图形,并使用它重新评估最佳已发布的文档级AMR解析器。我们还提出了一种与COREREFER解决系统的顶部组合的管道方法,为未来的研究提供了强大的基线。
translated by 谷歌翻译
我们提出了一种基于转换的系统来转换摘要意义代表(AMR)进入SPARQL,了解知识库问题应答(KBQA)。这允许将抽象问题的一部分委派给强训练的语义解析器,同时使用少量配对数据学习转换。我们从最近的工作相关的AMR和SPARQL构造,而不是应用一套规则,我们教导BART模型选择性地使用这些关系。此外,在最近的语义解析作品之后,我们避免在BART的注意机制中进行了显式编码AMR,而是编码解析器状态。结果模型很简单,为其决策提供支持文本,并且优于LC-Quad(F1 53.4)中的基于AMR的KBQA中的最新进展,在QAL(F1 30.8)中匹配,同时利用相同的归纳偏差。
translated by 谷歌翻译
由于包括架构改进和转移学习的效果,AMR Parsing在过去三年中经历了不起起的表现增加。自学习技术也在推动性能方面发挥作用。然而,对于最近的高性能解析器,自学和银数据生成的效果似乎褪色。在本文中,我们表明,通过将基于Spatch的集合技术与集合蒸馏组合来克服这一减少的银数据的递减。在一个广泛的实验设置中,我们首次推出超过85次Spatch以上的单一模型英语解析器性能并返回大量收益。我们还为中国,德语,意大利语和西班牙语进行了跨语态amr解析的新型最先进的。最后,我们探讨了所提出的蒸馏技术对领域适应的影响,并表明它可以产生竞争对QALD-9的人类注释数据的增益,并为生物群体实现新的最先进。
translated by 谷歌翻译
Detecting abrupt changes in data distribution is one of the most significant tasks in streaming data analysis. Although many unsupervised Change-Point Detection (CPD) methods have been proposed recently to identify those changes, they still suffer from missing subtle changes, poor scalability, or/and sensitive to noise points. To meet these challenges, we are the first to generalise the CPD problem as a special case of the Change-Interval Detection (CID) problem. Then we propose a CID method, named iCID, based on a recent Isolation Distributional Kernel (IDK). iCID identifies the change interval if there is a high dissimilarity score between two non-homogeneous temporal adjacent intervals. The data-dependent property and finite feature map of IDK enabled iCID to efficiently identify various types of change points in data streams with the tolerance of noise points. Moreover, the proposed online and offline versions of iCID have the ability to optimise key parameter settings. The effectiveness and efficiency of iCID have been systematically verified on both synthetic and real-world datasets.
translated by 谷歌翻译
Training of a Machine Learning model requires sufficient data. The sufficiency of the data is not always about the quantity, but about the relevancy and reduced redundancy. Data-generating processes create massive amounts of data. When used raw, such big data is causing much computational resource utilization. Instead of using the raw data, a proper Condensed Representation can be used instead. Combining K-means, a well-known clustering method, with some correction and refinement facilities a novel Condensed Representation method for Machine Learning applications is introduced. To present the novel method meaningfully and visually, synthetically generated data is employed. It has been shown that by using the condensed representation, instead of the raw data, acceptably accurate model training is possible.
translated by 谷歌翻译
Speech translation (ST) is the task of directly translating acoustic speech signals in a source language into text in a foreign language. ST task has been addressed, for a long time, using a pipeline approach with two modules : first an Automatic Speech Recognition (ASR) in the source language followed by a text-to-text Machine translation (MT). In the past few years, we have seen a paradigm shift towards the end-to-end approaches using sequence-to-sequence deep neural network models. This paper presents our efforts towards the development of the first Broadcast News end-to-end Arabic to English speech translation system. Starting from independent ASR and MT LDC releases, we were able to identify about 92 hours of Arabic audio recordings for which the manual transcription was also translated into English at the segment level. These data was used to train and compare pipeline and end-to-end speech translation systems under multiple scenarios including transfer learning and data augmentation techniques.
translated by 谷歌翻译
Ordinary Differential Equations (ODE)-based models have become popular foundation models to solve many time-series problems. Combining neural ODEs with traditional RNN models has provided the best representation for irregular time series. However, ODE-based models require the trajectory of hidden states to be defined based on the initial observed value or the last available observation. This fact raises questions about how long the generated hidden state is sufficient and whether it is effective when long sequences are used instead of the typically used shorter sequences. In this article, we introduce CrossPyramid, a novel ODE-based model that aims to enhance the generalizability of sequences representation. CrossPyramid does not rely only on the hidden state from the last observed value; it also considers ODE latent representations learned from other samples. The main idea of our proposed model is to define the hidden state for the unobserved values based on the non-linear correlation between samples. Accordingly, CrossPyramid is built with three distinctive parts: (1) ODE Auto-Encoder to learn the best data representation. (2) Pyramidal attention method to categorize the learned representations (hidden state) based on the relationship characteristics between samples. (3) Cross-level ODE-RNN to integrate the previously learned information and provide the final latent state for each sample. Through extensive experiments on partially-observed synthetic and real-world datasets, we show that the proposed architecture can effectively model the long gaps in intermittent series and outperforms state-of-the-art approaches. The results show an average improvement of 10\% on univariate and multivariate datasets for both forecasting and classification tasks.
translated by 谷歌翻译
Predicting the health risks of patients using Electronic Health Records (EHR) has attracted considerable attention in recent years, especially with the development of deep learning techniques. Health risk refers to the probability of the occurrence of a specific health outcome for a specific patient. The predicted risks can be used to support decision-making by healthcare professionals. EHRs are structured patient journey data. Each patient journey contains a chronological set of clinical events, and within each clinical event, there is a set of clinical/medical activities. Due to variations of patient conditions and treatment needs, EHR patient journey data has an inherently high degree of missingness that contains important information affecting relationships among variables, including time. Existing deep learning-based models generate imputed values for missing values when learning the relationships. However, imputed data in EHR patient journey data may distort the clinical meaning of the original EHR patient journey data, resulting in classification bias. This paper proposes a novel end-to-end approach to modeling EHR patient journey data with Integrated Convolutional and Recurrent Neural Networks. Our model can capture both long- and short-term temporal patterns within each patient journey and effectively handle the high degree of missingness in EHR data without any imputation data generation. Extensive experimental results using the proposed model on two real-world datasets demonstrate robust performance as well as superior prediction accuracy compared to existing state-of-the-art imputation-based prediction methods.
translated by 谷歌翻译
在尚未解决反事实解释的挑战中(CE),存在稳定性,各种CE的综合以及缺乏合理性/稀疏性保证。从更实用的角度来看,最近的研究表明,规定的反事实回复通常并非完全由个人实现,并证明大多数最先进的CE算法在这种嘈杂的环境中很可能会失败。为了解决这些问题,我们提出了一个概率框架,为每个观察结果提供了稀疏的本地反事实规则:我们提供的规则可以提供一系列可以用给定的高概率改变决策的价值观,而不是给出不同的CE。此外,通过构造从这些规则中得出的回报是可靠的。这些本地规则被汇总为区域反事实规则,以确保跨观察结果的反事实解释的稳定性。我们的本地和区域规则保证了recourse忠实于数据分布,因为我们的规则使用一致的估计器对基于随机森林的决定的概率进行了始终如一的估计。此外,当我们选择具有更改决策概率的最小变量时,这些概率给出了可解释和稀疏的规则。可以使用计算反事实规则的代码,我们将其相关性与标准CE和最近的类似尝试进行比较。
translated by 谷歌翻译